Skip to content

Removing flakiness using .json() to check JSON string responses #96

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

akshathphillips
Copy link

What is the purpose of this PR

  • This PR fixes the error resulting from a flaky test: com.didispace.chapter21.Chapter21ApplicationTests
  • The mentioned test may fail or pass without changes made to the source code when it is run depending on the JSON's non-deterministic deserialization.

Why the tests fail

  • This test fails because when the JSON's response is deserialized to a String.class, the order of the fields inside the object may change to be in random positions. For example, {"id":1, "age":20}, or {"age":20, "id":1}

Reproduce the test failure

  • Run the tests with NonDex maven plugin. The commands to recreate the flaky test failures are:
    • Run the test with the NonDex maven plugin
      • mvn -pl 2.x/chapter2-1 test edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=com.didispace.chapter21.Chapter21ApplicationTests#testUserController -DnondexRuns=10
  • Fixing the flaky test now may prevent flaky test failures for users checking out this project.

Expected results

  • The test should run successfully when run with NonDex.

Actual Result

We get the following failure for test com.didispace.chapter21.Chapter21ApplicationTests

[ERROR] Failures: 
[ERROR]   Chapter21ApplicationTests.testUserController:52 Response content
Expected: "[{\"id\":1,\"name\":\"测试大师\",\"age\":20}]"
     but: was "[{\"age\":20,\"id\":1,\"name\":\"测试大师\"}]"

Fix

For test com.didispace.chapter21.Chapter21ApplicationTests - Parse the expected and actual strings as JSON and assert the two are "similar" - i.e. they contain the same attribute-value pairs regardless of formatting with a lenient checking. The .json(), does a lenient checking on the fields, which solves this issue.

…rt the two are "similar" - i.e. they contain the same attribute-value pairs regardless of formatting with a lenient checking
@xiaofeiahaha
Copy link

xiaofeiahaha commented May 2, 2023 via email

@liuhamglin
Copy link

liuhamglin commented May 2, 2023 via email

@loquitowen
Copy link

loquitowen commented May 2, 2023 via email

@akshathphillips
Copy link
Author

Hello. Any updates on this?

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants